Repository navigation
Implement issue labeller bot - #11615
Conversation
This template exactly matches what the bot will parse.
Spoiler: it can't as of this commit. But it should.
This makes it much easier to test.
I was just being dumb. Obviously we want the functions _inside_ the returned context to be Sinon stubs, not the factory function itself.
|
Introduced ESLint to do linting/style checking - it's possible it could be a little better, but I think it's good for now. |
cd39d1a to
22d930e
Compare
|
Ah yeah, sorry @Hainish - upstream docs are a little scarce but it looks like if you set |
|
Looks like it has crashed again. Thanks for the tip @strugee - I've stopped the bot and removed the issue template for the time being. I'll re-enable it once we resolve a public EFF subdomain to the bot and start running it that way. |
|
OK. Keep me posted. |
|
Waiting on our internal techops team to do some server management before running bot again. |
|
The issue labeller bot is re-activated, and I've incorporated a (slightly) modified version of @jeremyn's suggested |
|
I just changed the issue template to use a different style for comments, which I think looks much better. |
|
The bot's behavior in issue #12956 isn't great, with four requests for clarification in one day. I think the bot should ask for clarification at most one time. |
|
So sorry all - I've been swamped with midterm papers due this week. I'll try to fix the bug as soon as I get the time. |
|
The bot seems to have crashed - here's the log: Error: No X-Hub-Signature found on request
at hasError (/opt/https-everywhere/utils/issue-format-bot/node_modules/github-webhook-handler/github-webhook-handler.js:46:17)
at handler (/opt/https-everywhere/utils/issue-format-bot/node_modules/github-webhook-handler/github-webhook-handler.js:57:14)
at Server.http.createServer (/opt/https-everywhere/utils/issue-format-bot/node_modules/probot/lib/server.js:5:5)
at emitTwo (events.js:125:13)
at Server.emit (events.js:213:7)
at parserOnIncoming (_http_server.js:602:12)
at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
Error: No X-Hub-Signature found on request
at hasError (/opt/https-everywhere/utils/issue-format-bot/node_modules/github-webhook-handler/github-webhook-handler.js:46:17)
at handler (/opt/https-everywhere/utils/issue-format-bot/node_modules/github-webhook-handler/github-webhook-handler.js:57:14)
at Server.http.createServer (/opt/https-everywhere/utils/issue-format-bot/node_modules/probot/lib/server.js:5:5)
at emitTwo (events.js:125:13)
at Server.emit (events.js:213:7)
at parserOnIncoming (_http_server.js:602:12)
at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
07:50:07.370Z DEBUG PRobot: Refreshed Alexa rankings.
npm info lifecycle issue-format-bot@1.0.0~poststart: issue-format-bot@1.0.0
npm info ok |
|
I've restarted the service for now. @jeremyn has noted that the bot has had limited utility. I can shut it down again or we can try to improve it. I'm interested in community input on this. |
|
The bot would be helpful as part of a larger automated process, for example in migrating simple "add this domain" issues to the wiki or some collection issue or something else, like we discuss in issue #13890. The Alexa labeling is helpful. I hope the current bot is an improvement over @Hainish 's previous process. The required The bot only processes new issues, but we have a huge issue backlog, which means that even the best-performing bot will only process a small percentage of open issues. Probably more extensive bot actions, if we want to have them, should wait until the backlog has been reduced. I would be okay with reducing the bot's behavior to the Alexa labeling for now. |
|
Pinging @strugee for input as well, since he reached out to me recently to check up on the bot and how it has worked out. |
|
@jeremyn my previous process for issues was nonexistent. For PRs, the labeller is still behaving as it was before the creation of the issue labeller bot. Unlike the issues bot, it did dig through the PR backlog, but this is because it was easy to do so: you can simply parse the PR to see what its targets are. For issues, things are a little more complicated. Without the issue template, it's hard for a bot to parse common vernacular for what is actually being requested. |
|
I should add that I appreciate @strugee 's willingness to contribute. Also, while the bot has had some technical problems, that is to be expected with new software and I don't mean to criticize @strugee 's skill as a programmer. My complaints (if I am complaining) are related to whether the bot is a good fit for the project, or whether we are putting it to good use. |
|
@Hainish In that case it would be nice to merge the issue bot and the Alexa bot together so there is less code hanging around. We might re-envision the issue bot as a broader "GitHub bot" and in that way find more stuff to do with it. |
|
Thanks @jeremyn - that's probably a good to-do item. |
|
@Hainish that stack trace is being thrown in a Probot dependency, not our custom code. Apparently someone has POSTed to the API endpoint and the code is complaining because the code doesn't contain the usual signature from GitHub. The fact that it's throwing instead of just dropping the request is a bug in Probot (the bot framework). We should try just upgrading Probot and see if it's still reproducible (assuming of course that my theory is correct and it is reproducible). Otherwise someone could file a bug or PR upstream. |
|
As for the usefulness, I agree it doesn't do as much as it could. The original thinking behind the @jeremyn what do you mean reducing the functionality to Alexa labeling? AFAIK that's basically all it does right now. I mean it says some other stuff but most of that is handling user error. Also I too have always thought it felt kinda weird to have a separate bot for PRs. (Also I appreciate you saying the bit about not criticizing me, but absolutely don't worry! I didn't take it that way at all 👍) Here's what I would suggest for some broad TODOs, in no particular order:
I also wonder if @brainwane has any thoughts on this since I believe she's been looking into similar workflow questions? |
|
@strugee Building consensus and then acting on it is a serious challenge for this project. Probably the bot shouldn't do anything unless it is directly supporting something that already has overwhelming support from the community, or unless the EFF has some firm vision it's willing to support. Implementing something that only might be used ( My earlier comment was confused, I thought that the issue bot was also handling Alexa labeling for PRs. Handling old issues is a problem and not one that a bot can solve. I opened issue #13890 in December to talk about it but it hasn't really gone anywhere (see above about building consensus). I don't think anyone thinks the required fields are very useful right now. I would disable them entirely. |
|
@jeremyn well,
Gotcha, no worries.
Well, the bot could probably help. At least then we'd get the Alexa labeling for old issues which would help with triage. Some things you mention too could help, e.g.:
This could be helped by using
|
|
@brainwane and I will be drafting some specs for tool and workflow improvements. We'll be sure to review this conversation and summarize into those specs, then sharing for comment. |
|
@strugee The bot could help with old issues, but probably won't, unless people agree on what it could do to help and the bot then does those things. The agreeing step is the problem. |
I mean, I was assuming it'd do the same thing it does for new issues...? That is, give Alexa labels and (possibly) label based on |
|
For Alexa labelling the old issues, it probably makes more sense for someone to go through the list manually. |
Reached a point where the bot's basic functionality works pretty well, so I thought I'd open a WIP PR for review.
Note that if you can think of any other weird input that users might give, I'd love to expand the test suite with that type of stuff.
Still TODO:
:s in them):s in ittop-Xlabels before relabelingRef #6322